Exchange 2007 stops responding. No sign of errors in the logs
Hi,
Until recently our Exchange 2007 server has been responding fine. It's just sat in the server room and done it's thing. Recently it's started to beomc unresponsive over night. When I examine it in this state you can interact with it locally
but it doesn't appear on the network. A reboot fixes this. However, although the server responds normally at this point, users periodically complain that the server will stop responding for a few seconds.
Checking the logs not much not much that's interesting - with a couple of exceptions. I've run BPA and again nothing.
All that I can find in the log is this:
Log Name: Application
Source: MSExchange ADAccess
Date: 11/05/2010 14:33:19
Event ID: 2159
Task Category: Validation
Level: Warning
Keywords: Classic
User: N/A
Computer: Exchange.SFQCHome.sfqc.co.uk
Description:
Process w3wp.exe (OWA) (PID=4084). Configuration object CN=All Contacts,CN=All Address Lists,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=SFQCHome,DC=sfqc,DC=co,DC=uk read from SFQCNT5.SFQCHome.sfqc.co.uk failed validation and will be excluded from the result set. Set event logging level for Validation category to Maximum to get additional events about each failure.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchange ADAccess" />
<EventID Qualifiers="32772">2159</EventID>
<Level>3</Level>
<Task>6</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2010-05-11T13:33:19.000Z" />
<EventRecordID>106472</EventRecordID>
<Channel>Application</Channel>
<Computer>Exchange.SFQCHome.sfqc.co.uk</Computer>
<Security />
</System>
<EventData>
<Data>w3wp.exe (OWA)</Data>
<Data>4084</Data>
<Data>CN=All Contacts,CN=All Address Lists,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=SFQCHome,DC=sfqc,DC=co,DC=uk</Data>
<Data>SFQCNT5.SFQCHome.sfqc.co.uk</Data>
</EventData>
</Event>
Log Name: Application
Source: MSExchange ADAccess
Date: 11/05/2010 14:33:19
Event ID: 2161
Task Category: Validation
Level: Warning
Keywords: Classic
User: N/A
Computer: Exchange.SFQCHome.sfqc.co.uk
Description:
Process w3wp.exe (OWA) (PID=4084). Object CN=All Contacts,CN=All Address Lists,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=SFQCHome,DC=sfqc,DC=co,DC=uk read from SFQCNT5.SFQCHome.sfqc.co.uk failed validation. Attribute: Container. Error message: Cannot calculate value of property "Container": "Cannot get the DN of depth -1 from DN CN=All Contacts,CN=All Address Lists,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=SFQCHome,DC=sfqc,DC=co,DC=uk.".. Invalid data: All Contacts.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchange ADAccess" />
<EventID Qualifiers="32772">2161</EventID>
<Level>3</Level>
<Task>6</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2010-05-11T13:33:19.000Z" />
<EventRecordID>106471</EventRecordID>
<Channel>Application</Channel>
<Computer>Exchange.SFQCHome.sfqc.co.uk</Computer>
<Security />
</System>
<EventData>
<Data>w3wp.exe (OWA)</Data>
<Data>4084</Data>
<Data>CN=All Contacts,CN=All Address Lists,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=SFQCHome,DC=sfqc,DC=co,DC=uk</Data>
<Data>SFQCNT5.SFQCHome.sfqc.co.uk</Data>
<Data>Container</Data>
<Data>Cannot calculate value of property "Container": "Cannot get the DN of depth -1 from DN CN=All Contacts,CN=All Address Lists,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=SFQCHome,DC=sfqc,DC=co,DC=uk.".</Data>
<Data>All Contacts</Data>
</EventData>
</Event>
Has anyone seen this happen before?
And does anyone know how to fix these AD errors?
May 11th, 2010 4:40pm
DId you run ExBpa against the server for both health and permissions?
When do backups or AV scans run?
Free Windows Admin Tool Kit Click here and download it now
May 11th, 2010 4:58pm
Yes looked at ExBpa (as noted above). Nothing is showing up.
Today I am getting the following two errors:
Log Name: Application
Source: MSExchange ADAccess
Date: 12/05/2010 14:19:46
Event ID: 2102
Task Category: Topology
Level: Error
Keywords: Classic
User: N/A
Computer: Exchange.xxxHome.xxx.co.uk
Description:
Process MSEXCHANGEADTOPOLOGYSERVICE.EXE (PID=1880). All Domain Controller Servers in use are not responding:
xxxNT5.xxxHome.xxx.co.uk
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchange ADAccess" />
<EventID Qualifiers="49156">2102</EventID>
<Level>2</Level>
<Task>3</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2010-05-12T13:19:46.000Z" />
<EventRecordID>107885</EventRecordID>
<Channel>Application</Channel>
<Computer>Exchange.xxxHome.xxx.co.uk</Computer>
<Security />
</System>
<EventData>
<Data>MSEXCHANGEADTOPOLOGYSERVICE.EXE</Data>
<Data>1880</Data>
<Data>xxxNT5.xxxHome.xxx.co.uk
</Data>
</EventData>
</Event>
And
Log Name: Application
Source: MSExchange System Attendant Mailbox
Date: 12/05/2010 13:35:14
Event ID: 4001
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: Exchange.xxxHome.xxx.co.uk
Description:
A transient failure has occurred. The problem may resolve itself in awhile. The service will retry in 56 seconds. Diagnostic information:
Could not find any available Domain Controller.
Microsoft.Exchange.Data.Directory.ADTransientException: Could not find any available Domain Controller.
at Microsoft.Exchange.Data.Directory.ConnectionPoolManager.GetConnection(ConnectionType connectionType, ADObjectId domain, String serverName, Int32 port, NetworkCredential credential)
at Microsoft.Exchange.Data.Directory.ConnectionPoolManager.GetConnection(ConnectionType connectionType)
at Microsoft.Exchange.Data.Directory.ADSession.GetConnection(String preferredServer, Boolean isWriteOperation, Boolean isNotifyOperation, ADObjectId& rootId)
at Microsoft.Exchange.Data.Directory.ADSession.GetReadConnection(String preferredServer, ADObjectId& rootId)
at Microsoft.Exchange.Data.Directory.ADSession.Find(ADObjectId rootId, String optionalBaseDN, ADObjectId readId, QueryScope scope, QueryFilter filter, SortBy sortBy, Int32 maxResults, IEnumerable`1 properties, CreateObjectDelegate objectCreator,
CreateObjectsDelegate arrayCreator)
at Microsoft.Exchange.Data.Directory.ADSession.Find(ADObjectId rootId, QueryScope scope, QueryFilter filter, SortBy sortBy, Int32 maxResults, IEnumerable`1 properties, CreateObjectDelegate objectCtor, CreateObjectsDelegate arrayCtor)
at Microsoft.Exchange.Data.Directory.ADSession.Find[TResult](ADObjectId rootId, QueryScope scope, QueryFilter filter, SortBy sortBy, Int32 maxResults, IEnumerable`1 properties)
at Microsoft.Exchange.Data.Directory.SystemConfiguration.ADSystemConfigurationSession.Find[TResult](ADObjectId rootId, QueryScope scope, QueryFilter filter, SortBy sortBy, Int32 maxResults)
at Microsoft.Exchange.Data.Directory.SystemConfiguration.ADSystemConfigurationSession.FindServerByFqdn(String serverFqdn)
at Microsoft.Exchange.Data.Directory.SystemConfiguration.ADSystemConfigurationSession.FindLocalServer()
at Microsoft.Exchange.Servicelets.SystemAttendantMailbox.Servicelet.Work()
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchange System Attendant Mailbox" />
<EventID Qualifiers="49152">4001</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2010-05-12T12:35:14.000Z" />
<EventRecordID>107822</EventRecordID>
<Channel>Application</Channel>
<Computer>Exchange.xxxHome.xxx.co.uk</Computer>
<Security />
</System>
<EventData>
<Data>56</Data>
<Data>Could not find any available Domain Controller.</Data>
<Data>Microsoft.Exchange.Data.Directory.ADTransientException: Could not find any available Domain Controller.
at Microsoft.Exchange.Data.Directory.ConnectionPoolManager.GetConnection(ConnectionType connectionType, ADObjectId domain, String serverName, Int32 port, NetworkCredential credential)
at Microsoft.Exchange.Data.Directory.ConnectionPoolManager.GetConnection(ConnectionType connectionType)
at Microsoft.Exchange.Data.Directory.ADSession.GetConnection(String preferredServer, Boolean isWriteOperation, Boolean isNotifyOperation, ADObjectId& rootId)
at Microsoft.Exchange.Data.Directory.ADSession.GetReadConnection(String preferredServer, ADObjectId& rootId)
at Microsoft.Exchange.Data.Directory.ADSession.Find(ADObjectId rootId, String optionalBaseDN, ADObjectId readId, QueryScope scope, QueryFilter filter, SortBy sortBy, Int32 maxResults, IEnumerable`1 properties, CreateObjectDelegate objectCreator,
CreateObjectsDelegate arrayCreator)
at Microsoft.Exchange.Data.Directory.ADSession.Find(ADObjectId rootId, QueryScope scope, QueryFilter filter, SortBy sortBy, Int32 maxResults, IEnumerable`1 properties, CreateObjectDelegate objectCtor, CreateObjectsDelegate arrayCtor)
at Microsoft.Exchange.Data.Directory.ADSession.Find[TResult](ADObjectId rootId, QueryScope scope, QueryFilter filter, SortBy sortBy, Int32 maxResults, IEnumerable`1 properties)
at Microsoft.Exchange.Data.Directory.SystemConfiguration.ADSystemConfigurationSession.Find[TResult](ADObjectId rootId, QueryScope scope, QueryFilter filter, SortBy sortBy, Int32 maxResults)
at Microsoft.Exchange.Data.Directory.SystemConfiguration.ADSystemConfigurationSession.FindServerByFqdn(String serverFqdn)
at Microsoft.Exchange.Data.Directory.SystemConfiguration.ADSystemConfigurationSession.FindLocalServer()
at Microsoft.Exchange.Servicelets.SystemAttendantMailbox.Servicelet.Work()</Data>
</EventData>
</Event>
It seems that the exchange thinks the DC is not communicating - as though it's not on the network - and yet it definitely is. The two servers are actually located within the same server room and no other servers are reporting a problem. While
these errors occurred I had a RDP session open on the DC and it appeaered fine. No errors are being reported in it's event log either.
May 12th, 2010 4:59pm
So Exchange is not installed on a DC is it?
Does DNS look valid from the Exchange perspective? Settings correct?
You may want to run DCDiag and netdiag as well.
Free Windows Admin Tool Kit Click here and download it now
May 12th, 2010 6:17pm
No, Exchange is not installed on a DC. Best practise is not to do that.
DNS resoves fine. Not sure what you mean by settings - can you elaborate?
I've just installed RSAT on the server. FYI this is Exchange 2007 running on Srvr 2k8.
The DC passes all DCDiag tests. Not tried from Exchange yet.
DCdiag reports as follows (when run from the context of the Exchange)
Directory Server Diagnosis
Performing initial setup:
* Identified AD Forest.
Done gathering initial info.
Doing initial required tests
Testing server: Default-First-Site\xxxxNT5
Starting test: Connectivity
......................... xxxxNT5 passed test Connectivity
Doing primary tests
Testing server: Default-First-Site\xxxxNT5
Starting test: Advertising
......................... xxxxNT5 passed test Advertising
Starting test: FrsEvent
......................... xxxxNT5 passed test FrsEvent
Starting test: DFSREvent
......................... xxxxNT5 passed test DFSREvent
Starting test: SysVolCheck
......................... xxxxNT5 passed test SysVolCheck
Starting test: KccEvent
......................... xxxxNT5 passed test KccEvent
Starting test: KnowsOfRoleHolders
......................... xxxxNT5 passed test KnowsOfRoleHolders
Starting test: MachineAccount
......................... xxxxNT5 passed test MachineAccount
Starting test: NCSecDesc
......................... xxxxNT5 passed test NCSecDesc
Starting test: NetLogons
......................... xxxxNT5 passed test NetLogons
Starting test: ObjectsReplicated
......................... xxxxNT5 passed test ObjectsReplicated
Starting test: Replications
......................... xxxxNT5 passed test Replications
Starting test: RidManager
......................... xxxxNT5 passed test RidManager
Starting test: Services
......................... xxxxNT5 passed test Services
Starting test: SystemLog
......................... xxxxNT5 passed test SystemLog
Starting test: VerifyReferences
......................... xxxxNT5 passed test VerifyReferences
Running partition tests on : ForestDnsZones
Starting test: CheckSDRefDom
......................... ForestDnsZones passed test CheckSDRefDom
Starting test: CrossRefValidation
......................... ForestDnsZones passed test
CrossRefValidation
Running partition tests on : DomainDnsZones
Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom
Starting test: CrossRefValidation
......................... DomainDnsZones passed test
CrossRefValidation
Running partition tests on : Schema
Starting test: CheckSDRefDom
......................... Schema passed test CheckSDRefDom
Starting test: CrossRefValidation
......................... Schema passed test CrossRefValidation
Running partition tests on : Configuration
Starting test: CheckSDRefDom
......................... Configuration passed test CheckSDRefDom
Starting test: CrossRefValidation
......................... Configuration passed test CrossRefValidation
Running partition tests on : xxxxHome
Starting test: CheckSDRefDom
......................... xxxxHome passed test CheckSDRefDom
Starting test: CrossRefValidation
......................... xxxxHome passed test CrossRefValidation
Running enterprise tests on : xxxxHome.xxxx.co.uk
Starting test: LocatorCheck
May 12th, 2010 6:54pm
By checking DNS, I mean that its pointing to a valid DNS server, no errors in the on the DNS server side etc..
What do you see listed for EVent 2080?
http://support.microsoft.com/kb/316300
Free Windows Admin Tool Kit Click here and download it now
May 13th, 2010 8:52pm
Roles Enabled Reachability Synchronized GC capable PDC SACL right Critical Data Netlogon OS Version
CDG 1 7 7 1 0 1 1 7 1
Yes it appears to be pointing at the correct DNS server. Nothing showing up in the DNS log and name resolution appears to be ok.
I've examined event 2080 - thanks for that link.
This is the results:
Roles Enabled Reachability Synchronized GC capable PDC SACL right Critical Data Netlogon OS Version
CDG 1 7 7 1 0 1 1 7 1
Looks like everything has checked out. Except the PDC check - which is odd as there is only 1 DC on the network. Having looked at the AD users & comps snap-in I can see that this DC is listed as a role holder for everything including PDC
emulator.
BTW Exchange is now responding and seems to have stabilised. I'm not getting Event 26 in Outlook clients either. Very odd.
I'd still like to clean up the topology I'm just not sure how to do that.
May 14th, 2010 11:33am